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I INTRODUCTION (U) 


(U) Since 1973, when the investigations of the human information-accessing capability 
called remote viewing (RV) first began at SRI International,!* evaluating the quality of the 
information obtained has been a continuing challenge. In order to develop valid evaluation 


procedures, two basic questions must be addressed: 


(1) What constitutes the target? 


(2) What constitutes the response? 


__' If the RV task is research-oriented, the targets are known, ‘and therefore can be 
precisely defined. In 4ppicahon -oriented tasks, however, the targets are generally unknown 
and their descriptions are problematical. In both task domains, RV responses tend to consist of 
sketches and written phrases. A method to encode unambiguously this type of “natural 
language” is one of the unsolved problems in computer science, and there has been little progress 


to date. Thus, a complete definition of an RV response is also problematical. 


} An opplicet ton-oriented RV task poses further problems. Bigh- quality RV does 


not always provide useful cppiicat’'tn For example, the RV may provide % ‘additional support for 


information that has been verified from other sources, but provide no new information. In some 
cases, however, an overall low-quality RV may provide key elements that positively influence an 


analyst’s interpretation. 


Another characteristic of current laboratory analysis techniques is that they do 
not provide an a priori assessment of the RV quality. While this is not a problem in the 
laboratory, * ms . applications require such evaluation. An RV analyst cannot provide . 
ratings from the RV alone; rather, the analyst must provide a priori 
probabilities that individual RV-response elements (og concepts) are present at the target site. It 
remains the responsibility of an analyst to ‘determine whether such data are ultimately 


useful. 


Analysis of laboratory RV has been a major part of the ongoing Cognitive 
Sciences Program.2-? For FY 1989, we focused on the development of a prototype analysis 


system that would provide the needed a priori assessments for @ppIi cach'on tasking 7 


(U) References are at the end of this report. 
t weak. fan OM, . ra i 
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II METHOD OF APPROACH (U) 


The analysis of remote viewing (RV) data in an Gpe Wi cahion environment differs 
considerably from laboratory analysis. Most often, analysts have incomplete or no information 
about the target site and are required to provide a priori assessments of data gathered from RV 
sessions. In this section we outline a prototype analysis system for ' + RV that uses 
concepts from fuzzy set theory, historical archival data, and “templates” of typical/ ae 
targets. In addition, we apply this prototype system to an existing target pool as an illustration of 


the power of the technique. 


A. (U) Fuzzy Set Formalism 


\ A more complete description of the full fuzzy set formalism can be found in our 
literature..? For the purpose of this report, we have summarized that formalism in general terms 


that are not specific to either laboratory experiments or Applicata, tasking. 


1. (U) Construction of Target and Response Fuzzy Sets 


(U) A formal definition of a target and its associated RV response (i.e., the data 
obtained from an RV session) is necessary to any analysis system. To use the fuzzy set method, a 
universal set of elements is constructed on which target and response descriptions are based. 
These elements should contain descriptive aspects of the target material and incorporate items 
that typify responses from the intended viewers. This universal set should also be extendible 


(i.e., allow for additional items that may arise in the responses). - 


(U) In general, the task of an RV analyst is to assign a membership value (,) 
between 0 and 1 to each element in the universal set. The numerical value for each element in a 
response is assigned by the degree to which the analyst is convinced that the given element is 
present in that response. Membership values for target elements are assigned on the basis of the 


degree to which the elements contribute to the target description. 


i | In the laboratory, the targets are known, so that defining a universal set of 
elements is comparatively straightforward.®? In épplicacttoy tasks, however, defining a single 
universal set of elements that is appropriate for all operations is difficult. Because the usual 
a task is so highly mission—dependent, defining a single universal set of elements that is 


customized to that mission becomes easier. 
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a 
, The dppivcatian analyst, as opposed to an RV analyst, should construct such 
ae a list for each mission. While there may be considerable similarities between element lists for 
different missions, undoubtedly the lists will require specialization. In Section II~C below, we 
show the construction of one element list and how it can be applied to a set of 65{ =, 
a L targets. 
5 2. (U) Analysis of Complete Responses 


| Once an appropriate universal set of elements has been created, and fuzzy 
sets that define the target and the response have been specified, the comparison between them is 
straightforward. We have defined accuracy as the percent of the target material that is described 


correctly by a response. Likewise, we have defined reliability (of the viewer) as the percent of 


the response that is correct.6 Although in the laboratory it is required to provide a posterior 


probability estimates of the target-response match, in an agolicat'ims Setting, this may be less 
important. All that is usually necessary is to describe the accuracy and reliability for complete 


responses, and for individual target elements of interest. These quantities for the jth sessions are 
n 
>, WelRi 1 Tie 
k= 
cea ara (a) 
>, WeRj x 
kal 


and 


a 
D> WR OT) Te 
_ =i 


aj= 


(2) 


’ 


7 
> WxTj,x 
k=1 


where the sum over & is called the sigma count in fuzzy-set terminology, and is defined as the sum 
of the membership values (j:) for the elements of the response, the target, or their intersection, 
and n is the number of possible elements as defined by the element list. A fuzzy intersection is 
defined as the minimum of the intersecting fuzzy set membership values. In this version of the 


definitions, we have allowed for the possibility of weighting the membership values, Wk, to 


ad ey oe, 


ee 


7 


| 
{ 
i 


provide mission-defined relevances. 
(U) For the above calculation to be meaningful, the membership values for the targets 
must be similar in kind to those for the responses. For most mission-dependent specifications, 
this is generally not the case. The target membership values represent the degree to which a 
particular element is characteristic of the target, and the response membership values represent 


the degree to which the analyst is convinced that the given element is represented in the 
response. 
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(U) Until RV abilities can encompass the recognition of elements as well as their degree of 
target characterization, we are required to modify the target fuzzy set. An analyst must decide 
upon a threshold above which an element is considered to be completely characteristic of the 
target site. In fuzzy set theory, this is called an w-cut: a technique to apply a threshold to the rv 
values such that if the original value exceeds it, reassign the value to 1, otherwise set it to 0. In 
this way, the analyst’s subjectivity can be encoded in the response fuzzy set, and Equations 1 and 


2 remain valid. 


3. (U) Analysis of an Individual Element 


(U) Equations 1 and 2 can be simplified to provide an accuracy and reliability on an 
individual element basis instead for a complete response. For example, let N be the number of 
sessions against different targets that exist in a current archive for a specified viewer. Let € be an 
element in question (e.g., airport). Then the empirical probability that element e is in the target, 


given that the viewer said it was, is given by 


R(e ge 


N, “ (3) 


where Nc is the number of times that the individual was correct, and Nr is the number of times 
that element € was mentioned in the response. R(e) is also the reliability of the viewer. for that 


specified element. 


(U) To compute what chance guessing would be, we must know the occurrence rate 
of element ¢ in the N sessions. Let No be the actual number of times element € was contained in 


the N targets. Then the chance-guessing empirical probability is given by 
t 


BY 


Ro(€) can also be considered as the guessing reliability (i.e., the reliability that would 
be observed if the viewer guessed « during every session). The more R(e) > Ro(e), the more 


reliable the individual is for the specified element. 


: (U) The empirical probability that the viewer said element e, given that it was in the 


target, is given by 


A(e) is also the accuracy of the viewer for that specified element. 


(U) As a numerical example, suppose a single viewer participated in N = 25 sessions. 


Let e = “airport.” Further suppose that No = 5 of the targets actually contained an airport. 
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(U) 

Then, Ro(airport) = 0.20 is the chance probability (i.e., guessing airport during every session 
would only by 20 percent reliable). Assume that the viewer mentioned airport Nr = 6 times and 
was correct Nc = 4 times. Then this viewer’s reliability for airports is computed as R(airport) = 
0.67 > Ro(airport) = 0.20. The viewer’s accuracy for airports is computed as A (airport) = Nc/No 
= 0.80. Thus in this example, we can conclude that this viewer is reasonably accomplished. at 


remote viewing an airport. 


B. (U) Prototype Analysis System 


( , We assume that an‘ ae andlyst has constructed a mission-dependent 


_——ae’ 


: i 
universal set of elements. We further assume that there are a number of competing 
interpretations of the target site in question. } 
i 
i 


1. (U) Target Templates 


The first step in our prototype ‘analysis system is to define templates (i.e., 


general descriptions of classes of target types) of all compenne tect “etpertanons from the 


te 


universal set of elements. ( 


i ~~) a Exactly what the templates should represent is entirely dependent upon 
what kind of information is sought. Both the underlying universal set of elements and the 
berpiaree must be constructed to be rich enough to allow for the encoding of all the information 
oe That is, if neither the set of elements nor the templates can meaningfully 
represent information about, the tame+ __y - then it will be unreasonable to consider 
asking, (eievant queshms abd : ' the site. Furthermore, a certain 
amount of atomization is necessary because such division into small units provides the potential 
for interactions within the universal set of elements. If the profile of a a “facility consists of a 
single element, the template would be useless unless the response directly: stated that particular 
element; rather, the profile should be constructed from groups of elemental features ~ 
: +} 


( There are two different ways to generate target templates. The most 


straightforward technique is also likely to be the most unreliable, because it relies on the analyst’s 
judgment of a single target type. With this method, the analyst, who is familiar with the 
intelligence problem at hand, simply generates membership values for elements from the 
universal set of elements based upon his or her general knowledge. Given the time and 
resources, the best way to generate template membership values is to encode known targets that 
are closely related. 2 Each template p. is the average value 


across targets, and thus is more reliable. If it is known that some targets are more 
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“characteristic” of the target type than others, then a weighted average should be computed. In 
symbols, 


uf =, (4) 


where the sums are over the available targets that constitute the template, wk are the target 


weights, and the j1j,x are the assigned membership values for target k. 


2. (U) Archival Database 


A critical feature of an analysis system fog ',RV data is that along 
with the current RV data to be evaluated, the individual viewer's past performance on an 
element-by-element basis must also be included. For example, if a viewer has been relatively 


unsuccessful at recognizing } facilities, then ag »reference in the current data should not 
contribute much in the overall analysis. 


\ ’ As ground truth becomes available for each session, a performance database 
Should be updated for each viewer to reflect the new information This database should be a 


fuzzy set whose membership values for each element are the reliabilities computed from 
*Equation 3. 


3.  (U) Optimized Probability List 


C \. The goal of any ( ‘RV analysis system is to provide an a priori 
prioritized and weighted list of target possibilities that results from a single remote viewing that is 


sensitive to the performance history of the viewer. Assuming that a template exists for each of 


the possibld. : interpretations, an ana should adhere to the following protocol: 


(1) Analyze the RV data by assigning a membership value (1) for each element inthe 
universal set of elements. Each MM represents the degree to which the analyst is 
convinced that the particular element is included in the response. 

r ee aren Bp a 


——— 


(2) Construct a crisp set, Rc, as an a@-cut of the Original response set. By adopting a 
threshold of 0.5, for example, then the resulting crisp set contains only those 
elements that the analyst deems most likely as being present in the response. 


(3) Construct an effective response set, Re, aS Re = Re U Ra, where Ra is the reliability set 


drawn from the archival database. “\ 
a RR 
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(4) Using this effective response set, compute an accuracy and reliability in accordance 
with Equations 1 and 2. Then compute a figure-of-merit, Mj, for the jth competing 
interpretations as 


Mj=ajXrj. 


Of course, the accuracy and reliability use the effective response set from step 3 
above. 


(5) Order the Ms from largest to smallest value. Since the figures—of—merit range in value 


from 0 to 1, they can be interpreted as relative probability values for each of the 
alternative target possibilities. 


te 
By following such a protocol, an analyst can produce a list of target alternatives that is sensitive to 


the current remote viewing yet takes into consideration to the individual viewer's archival record. 


cy 
” 


Cc. (U) Partial Application of Analysis System to Existing Target Pool 


(U) We have used an existing target pool (developed under a separate program) as a test 


bed for the analysis system described above. 


1. (U) Criteria for Inclusion in the Target Pool 
Targets in this pool have the following characteristics: 


e@ Each target is within an hour and a half automobile drive of SRI International. 
e Each target simulates A | ~-—"""“~Asite _» interest. 


e Each target fits generally within one of five functional categories: Production, 
Recreation, Scientific, Storage, and Transportation. 


“@ Each target meets a consensus agreement of experienced RV monitors and 
analysts about inclusion in the pool. 


(U) The pool consists of 65 targets. Initially, they were divided into 13 groups of five 
targets each, where each group contained one target from each of five functional categories. By 
carefully organizing the targets in this way, the maximum possible functional difference of the 


targets within each group was ensured. Table 1 shows a numerical listing of these targets. 
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Table 1 


(U) Numerical Listing of Targets 


. Space Capsule 

. Coastal Battery 

. Bay Area Rapid Transit 
. Salt Refinery 

. Candlestick Park 

. Solar Observatory 
. Food Terminal 

. Pedestrian Overpass 
. Electrica] Plant 

. White Plaza 

. Space Shuttle 

. Coastal Battery 

. Train Terminal 

. Sawmill 

. Pond 

. Wind Tunnel 

. Grain Terminal 

. Submarine 

. Cogeneration Plant 
. Park 

. Linear Accelerator 
. Dump 


> (U) Fuzzy Set Element List 


. Pump Station 

. Ice Plant 

. Caves/Cliffs 

. Bevatron 

. Barn 

. Golden Gate Bridge 
. Modern Windmills 

. Baylands Nature Preserve 
. Gas Plant 

. Auto Wreckers 

. Fishing Fleet 

. Radio Towers 

. Vineyard 

. Pharmaceutical Laboratory 
. Toxic Waste Storage 
. Airport 

. Car Wash 

. Old Windmill 

. Nuclear Accelerator 
. Reservoir 

. Train Station 


| In FY 1°89, we developed a prototype analysis system for analyzing targets 


an: ~e=cnses inf 


emote viewings. A list of elements, based on target function (i.e., 


the ==i0n specification), is arranged in levels from relatively abstract (information poor) to the 


rela. =~ complex (information rich), Having levels of elements is advantageous in that each can 


be ve_cred separately in the analysis. 


(U) This universal set of elements (included as Appendix A) represents primary 


elex=cz in the existing target pool of 65 targets. 


The set was derived exclusively from this 


knovz 2zget pool. In an actual RV session, however, a viewer does not have access to the 


eleruszz ist, and thus is not constrained to respond within its confines. An accurate RV analysis 


mus: 2=ude any additional Gata that may be provided in the response; therefore, additional 


spac 223 been provided on the analysis sheets (see Appendix A) to include elements that are 


part := e response but not :nitially included as part of the universal set. 
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Figure 1. © ag Cluster Diagram fort ~~ : \, Targets 


(U) We used the technology cluster (i.e., number 4 in Figure 1) to apply Equation 4 
to construct a technology target template. Table 2 shows the targets in this cluster, where the 


horizontal lines indicate the subclustering within the technology group shown in Figure 1. 
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Table 2 
(U) Technology Cluster 


1 


Transformer Station 
Modern Windmills 
31. Electrical Plant 

41. Cogeneration Plant 
1 


1 
3. Satellite Dish 
3. Satellite Dishes 
8, Observatory 
28. Solar Observatory 


$1. 


val " : 
= . 


Pharmaceutical Laboratory 
63. Nuclear Accelerator 
43. Linear Accelerator 


48. Bevatron 


UNCLASSIFIED “ 


(U) Table 3 shows those elements that met or exceeded average membership values 


of 0.4 using Equation 4. 


Table 3 


(U) Principal Elements Contained in the Technology Template 


‘s ffiliation 
Modifier 
88 
99 


Commercial/Private 


Research/Experimentation 


Electricity/Radio 


Objects High Technology Electronics 
Restricted Access 


120 Wires/Cables 


Abstract 


Activity—Passive 
Ambiance—Indoor 
Ambiance—Manmade 
Ambiance—Outdoor 
Size—Medium 
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(U) As a self-consistency check, we included the technology template in the total 
target pool and recalculated the clusters. As expected, the technology template was included 


within the subgroup of targets 3 and 13, and well within the technology cluster as a whole. 


D. (U) General Conclusions 


The goal of this effort was to develop an analysis system that would prove 
effective in providing a priori assessments of premote viewing tasks. If the proper 
mission-dependent universal set of elements can be identified, then, using a viewer-dependent 
reliability archive, data from a single remote viewing can be used to prioritize a set of alternative 


target templates so as to chose the most likely one for the mission. 
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- | AN ANALYST’S GUIDE TO THE UNIVERSAL SET OF ELEMENTS (U) 


A. (U) Introduction 


(U) This appendix is intended to assist an analyst in using the universal set of elements 
shown in Appendix A. We developed six levels of elements ranging from relatively abstract 
(information poor) to the relatively complex (information rich), 


B. (U) Element Levels and Their Use 


(U) The task of the analyst is to assign a membership value between 0 and 1 to each 
individual element. For targets, a numerical value will be assigned on the basis of the presence 
or absence of each element in terms of functional importance. For responses, the numerical 
value will be assigned on the basis of the degree to which the analyst is convinced*that the 
element is contained in the response. 


(U) All subsequent commentary is referenced by the element numbers in Appendix A. 
Although each level may contain a number of elements, only those individual elements that may 
need explanation are listed below. 


1. (U) Element Level—Affiliation 


(U) “Affiliation” represents an advanced level of remote viewing functioning. 
Although we infrequently ‘observe this advanced functioning, the data are valuable, and, 
therefore, are included. Elements in this level can be assigned membership values by asking the 


question, “Who owns the target?” There are only three “affiliation” elements: 


(1) | Commercial/Private. 


(2) Government: Federal, state, or local governmental ownership (e.g., municipal 
utilities), but excluding military. 


(3) Military: military ownership as separate from the above governmental ownership 
(e.g., a Navy submarine). 
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2. (U) Element Level—Function 


(U) “Function” also represents an advanced level of remote viewing functioning, and 
it may represent the most important information with regard to overall function. Elements are 
assigned membership values by asking the question, “What is(are) the primary function(s) of the 
target?” There are 14 “function” elements, and a few require further explanation: 


(6) Distribution: the primary function is to receive and to transmit something (e.g., an 
electrical transformer station). 


(8) Extraction: as in the extraction of minerals from the ground. 
(11) Reception: the primary function is only to receive (e.g., a satellite tracking station). 


(13) Refining: the primary function is to refine a raw material into an intermediate or 
finished product (e.g., a saw mill). 


(16) Transmission: the primary function is only to transmit (e.g., a radio tower). 


3. (U) Element Level—Attributes 


(U) “Attributes” can be thought of as clarification for the “function” level. 

Elements are assigned membership values by asking a question similar to, “If the function of the 
, : rear Me “ 

target is production, then what is being produced?” There are 20 “attribute” elements, ‘and the 


following require further explanation: 


(18) Animals: animals only. 
(20) Biology: the study of living things in general. 
(21) Chemistry: also includes chemicals. 


(23) Ecology: symbiotic systems in nature, as in ecological zones (e.g., the Bay Lands 
Nature Preserve). 


(24) Energy: energy in a broad sense that also includes radio waves. 

(29) Nature/Natural: general natural objects (e.g., plants and animals). 

(32) Plants: plants only. 

(33) Space exploration: general, includes all experimentation done in space. 


Elements 18 and 32 are given a membership value if the target/response is specifically oriented to 


one item. Otherwise element 29 should be assigned a value. 
’ 


4. (U) Element Level—Modifiers 

(U) “Modifiers” can be thought of as a clarification of the “attributes” level. 

Elements are assigned membership values by asking a question similar to, “If the function of the 
target is production, and vehicles are being produced, then what kind of vehicles are they?” 


There are 36 “modifiers” elements, and only element 66 requires further explanation: 
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; (66) Symbiotic: symbiotic relationships not subsumed under natural or ecology (e.g., a 
2 cogeneration plant). : 


5. (WU) Element Level—Objects 


(U) “Objects” contains specific elements not necessarily related to function. 
. Elements are assigned membership values on the basis of the presence or absence of each object 
in terms of functional importance. There are 47 “objects” elements, and the following require 


further explanation: 


i (77) Catwalk: elevated walkway. 
; (79) Coastline: used only as coastline of an ocean. 
(88) High-Technology Electronics: silicon-based technology. 


' (95) Port/Harbor: port should be marked as in port of departure (e.g., airport, train 
: Station, seaport). 


; (116) Water-Bounded: only completely bounded bodies of water (e.g., pool or pond). 
(117) Water-Canal: manmade. 


(118) Water-Large Expanse: the San Francisco Bay should be marked as a large 
i. expanse. 


(119) Water-River: also includes stream. 


i 6. (U) Element Level—General/Abstract Items 


(U) This level contains the most abstract elements. There are 31 elements, and the 
i following require further explanation: 
(121) Activity-Active: predominant visually active (e.g., an accelerator is very active 


, electromagnetically, but would be considered passive, because there is little visual 
i activity); potential activity is considered as passive. 


(122) Activity-Passive: predominant visually passive (e.g., a ballpark is passive most of 
the time). ‘ 


(123) Activity-Flowing (Water, Air, etc.): can be natural (e.g. creek) or manmade. 
: (128) Ambience-Dangerous: perceived and/or physically dangerous. 
(140) Colorful: to be used only if especially characteristic. 
. (141) Modern: to be used only if especially characteristic. 
: (142) Odd/Surprising: to be used only if especially characteristic. 
(143) Qld: to be used only if especially characteristic. 
(144) Personnel-Few: 1 to 10 employees mpstly full-time. 
(145) Personnel~Many: 10 to 1000 employees mostly full-time. 


(146) Personnel-None: no full-time employees, but occasional human attention is 
allowed. 


(148) Size-Large (University Campus): represents a “campus” size area. 
(149) Size-Medium (Building): size of typical single buildings. 

(150) Size-Small (Human): typically, the size of a human (i.e., 6 feet) 
(151) Dull: to be used only if especially characteristic of the color. 
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